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APPARATUS, SYSTEM AND METHOD FOR DISPUTE RESOLUTION, 
REGULATION COMPLIANCE AND QUALITY MANAGEMENT IN 

FINANCIAL INSTITUTIONS 

5 RELATED APPLICATIONS 

The present invention relates to PCT patent application number 
PCT/IL02/00539 for METHOD, APPARATUS AND SYSTEM FOR 
CAPTURING AND ANALYZING INERACTION BASED CONTENT, filed 18 
. July 2002, to PCT patent application number PCT/ELO 1/00796 for SYSTEM 

10 AND METHOD FOR CAPTURING BROWSER SESSIONS AND USER 
ACTION, filed 24 August 2001, to PCT patent application number 
PCT/IL02/01042 for SYSTEM AND METHOD FOR VIDEO CONTENT 
ANALYSIS-BASED DETECTION, SURVEILLANCE, AND ALARM 
MANAGEMENT, filed 24 December 2002, to PCT patent application number 

15 PCT/IL02/00741 for RECORDING AND QUALITY MANAGEMENT 
SOLUTIONS FOR WALK-IN ENVIRONMENTS, filed 5 September 2002, and 
to PCT patent application number PCT/IL03/00097 titled METHOD AND 
APPARATUS FOR VIDEO FRAME SEQUENCE-BASED OBJECT 
TRACKING filed 6 February 2003 , the content of which is hereby incorporated 

20 by reference. 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 
Hie present invention generally relates to multimedia recording and 
25 retrieval. More specifically, the present invention relates to a method, an 
sppamtus and a system for quality management, regulation compliance, and 
business intelligence in financial institutions, in particular. 
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DISCUSSION OF THE RELATED ART 



Financial institutes are operate under strict laws, rules and regulations 
designed to protect both the financial institutes and investors from fraudulent 
financial activities, such as money laundering, terror funding, theft, misconduct 
5 and other similar wrongdoing. 



period of time for the purposes of regulation compliance and for tiie resolution of 
dispute situations. A dispute is typically resolved by the re-construction of the 

10 transaction-specific interactions among the parties. Such re-construction is 
typically accomplished by obtaining one or more recordings representing the 
transaction-specific interactions, and by re-playing recordings- 

The completion of certain financial transactions may involve the 
performance of multiple interactions between the participating parties. A multiple 

15 interaction refers to the performance of more than one kiteraction during the same 
transaction. Multiple interactions may be of the same interac \ . . ^ or could 
* include different interaction types. One example, of a multiple . jon type is 
a conversation between two parties that involves capturing of voice over IP and 
instant messaging between the same parties, tiie interaction resulting in a 

20 transaction. 

Presently available recording systems have several disadvantages. The 
major drawback of existing systems concem the complete and total separation of 
the various recording sub-systems and the resulting complete lack of automatic 
connectivity options there between. Each interaction type associated with a 

25 specific communication channel is handled separately by a specific recording 
sub-system. Presently, there is no connectivity and users must search manually 
through the various archives of the separate sub-systems and attempt manually to 
associate interaction records to a relevant transaction. When a dispute situation 
arises there is a need to locate, to obtain, to connect and to integrate the entire set 

30 of interaction types captured during the performance of the transaction. 



The regulations require that financial institutes keep the records of the 
various interactions carried out between various personnel for a pre-specified 
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A second drawback of the existing systems and methods relate to the 
security of the information as presently no mechanisms exists, to prevent 
unauthorized persons to gam access to the information. 

A further drawback of the currently operating systems concern the lack of 
5 time synchronization option among the separate sub-systems and therefore the 
lack of time synchronization between the interactions recorded therein. The 
temporal order of the interactions is particularly important in the processing of a 
transaction and the time synchronization of the hiteractions associated with a 
specific transaction is essential to the resolving of 2l dispute situation. Existing 
10 systems do not provide for multimedia interaction handling in such a manner that 
all interactions are synchronized and played back in the correct sequence of the 
events 

Financial institutes attemptkig to comply with the regulations face 
additional problems. One problem regards the fact that the majority of the mitial 

15 interactions in a trading floor do not necessarily lead up to an actual financial 
transaction. It is an additional drr * the currently existing recording 

I systems that presently no differential r ^ Nation can be made between inforaial 
calls and calls that lead up to a transaction. 

Presently, each financial transaction stored m the bank's dealing systems is 

20 assigned a unique identifier in a manual manner. In many cases the terms of the 
transaction are specified during the interactions, but the actual entering of the 
transaction into the bank's dealing systems is usually made only after the 
mteraction is terminated. As a result the identifier assigned to the transaction in 
the bank's system has not yet existed at that point in time where the interaction 

25 was performed. The lack of connectivity and association between the transactions 
recorded in the bank's dealing systems and the interactions through which the 
transaction was originated often forces, internal auditors to examine large 
volumes recorded interactions manually until a suitable call is located. By 
allowing accurate location of the interaction records related to each transaction. 
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the same auditing teams using the same resources could be more efficient and 
audit more transactions. 

In addition, financial mstitution archives storing recorded customer- 
specific interactions contain a plurality of confidential information. Today, 

5 although financial institutes make substantial efforts to limit access to these 
archives there is often an urgent need to locate customer-specific interactions. 
Existing systems do not provide an automated solution for playback request 
handling while maintaining secrecy, privacy and controlled access to interactions. 
Financial institutions are also required to operate in accordance with specifically 

10 pre-defined rules of conduct. Supervisors must examine the interactions in order 
to make sure that the rules of conduct are implemented. Currently no efficient 
method exists to detect cases where the rales of conduct are not followed except 
through manual handling of a vast amoimt of phone calls or recordings. 

The rules of conduct recommend that details of all telephone deals should be 

15 confirmed by fax or by similar means however there is no efficient method 
presently tr :iie presence or the lack of the suitable confirmation 

v- procedures : a imstructured manual access to the archives and listening to 

vast amount of calls or recordings. 

Day trading for personal accounts is one example of a situation that requires 

20 fill! disclosure and transparency in order to ensure that traders give undivided 
attention to their institution's business, without being distracted by personal 
conflict of interests. There is no method currently to detect such cases other than 
unstructured manual access to the archives and listening to vast amount of calls or 
recordings, 

25 It would be readily perceived by one with ordinary skills in the art that the 

current systems do not provide efficient solutions for financial institutions. 
Therefore, there is need for a novel mtegrated multimedia capturing and logging 
system for financial institutions. Such a system should preferably handle all the 
applicable media type interactions and should retain the various media type 

30 interactions in an integrative, synchronized and associated manner. The captured 
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and logged elements would preferably comprise precisely timed and synchronized 
sequences of events in order to assist in the proper resolution of dispute 
situations, in dispute settlement and in the general identification of misconduct. 
The system would preferably include advanced tools, such as audio and data 
5 content analysis for the processing of the recorded mteractions for the purposes of 
regulation compliance, dispiite resolution, and advanced content analysis. 

SUMMARY OF THE PRESEKT nsrV^^ 
One aspect of the present invention regards an apparatus for the 

10 synchronization, integration and association of financially related captured 
interactions, the apparatus comprising a first interaction capturing device for 
capturing an interaction, a second interaction capturing device for capturing a 
second int^ction, a synchronizing device to synchronize the first interaction 
Avith the second interaction and an integration device to integrate the first 

15 interaction witii the second interaction into an integrated interaction. The 
apparatus further includes an association device to associate the integrated 
interaction with the transaction. The apparatus further includes a recording and 
logging device for storing the at least first interaction, the at least one second 
interaction. The apparatus fiirther includes an interaction retrieval device to 

20 retrieve the at least one first interaction, the at least one second interaction, and 
the integrated interaction. The apparatus further includes an interaction playback 
device to playback the at least one first interaction, the at least one second 
interaction and the integrated interaction. The apparatus further include a set-up 
and calibration device to set up the parameters of the apparatus and to adaptively 

25 calibrate the parameters of the apparatus. According to the invention the 
apparatus further including a secure access device to provide for secure, 
controlled and limited access to the at least one first interaction, to the at least 
one second interaction and to the integrated interaction. The secure access 
device further provides a secure time-based access. The apparatus further 

30 include a database for storing the at least one first interaction. The apparatus 
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further includes a database for storing the at least one second interaction. The 
apparatus further includes a database for storing the at least one first and second 
interaction. The apparatus further including a database for storing meta-data to 
provide the ability to retrieve the integrated interaction. The apparatus further 
5 includes a content analysis module to anal3^e the content of the at least one first 
interaction, the content of the at least one second interaction and the content of 
tiie integrated interaction. The apparatus further including at least one 
knowledge database for storing information associated with the at least one first 
interaction or with tiie at least one second interaction or with the mtegrated 
10 interaction. According to the invention the apparatus further including a 
notification message-generating device for generating a notification message 
associated with the at least one first interaction or with the at least one second 
interaction or mth the integrated interaction. The apparatus further including a 
notification message-generating device for generating a notification message 
15 associated with misconduct The apparatus further including a notification 
message-generating device for generating a notification message ' ; . with 
v; dispute resolution. The apparatus further including a notificn:: . .sage- 
generating device for generating a notification message associated with a request 
for playback. The apparatus further including a busmess analysis device to 
20 collect and analyze business performance and information. According to the 
mvention the apparatus further includes a monitoring device to monitor the 
content of the at least one first intCTaction or the at least one second interaction. 
The apparatus further including an evaluations device to evaluate the 
performance or level of service of a representative. The apparatus further 
25 comprising a surveillance device to detect and monitor a behavior pattem of a 
representative. The surveillance device provides notification regarding the 
detection of or changes in behavior patterns. The at least one first interaction and 
at least one second interaction are associated with transaction. The secure access 
to the at least one first interaction, the at least one second interaction and the 
30 integrated interaction is based on password-protected permission granting 
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process. The secure access is a password based access. The secure access is a 
password and time based access. The at least one first interaction, the at least 
one second interaction and the integrated interaction represent an interaction 
between at least two parties. The monitoring device, the surveillance device and 
5 the content analysis device provide information in real or off line. The apparatus 
further including an automatic scheduler device for the automatic selection of 
interactions to be monitored, analyzed and evaluated. The at least one first 
interaction or the at least one second interaction is an audio or video interaction. 
The at least one first interaction or the at least one second interaction is a data 
10 exchange or data capture interaction. 

A second aspect of the present invention regards to a method for the 
synchronization, integration and association of financially related captured 
interactions, the method comprising the steps of capturing an at least one first 
interaction at a first interaction capturing device, capturing an at least one second 
15 interaction at a second interaction capturmg device, synchronizing the at least 
one first interaction with the at least one . : :;:eraction and integrating the at 
I least one first interaction and the at 1^5 second interaction into one 

integrated interaction. The method fiirfher comprises the step of associating the 
integrated interaction with transaction. The method further comprises the step of 
20 recording and logging tiie at least one first interaction and the at least one second 
interaction on recording and logging device. According to the invention the 
method further comprises the step of retrieving the at least one first mteraction, 
the at least one second interaction. The method further comprises the step of 
generating the integrated interaction firom the retrieved at least one and second 
25 interaction. The method further comprises the step of playing back the at least 
one first interaction, the at least one second interaction, or the integrated 
interaction. The method further comprises the step of setting up the operating 
parameters of the method and adaptively calibrating the operative parameters. 
According to the invention the method further comprises the step of analyzing 
30 the contents of the at least one first interaction, the content of the at least one 
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second interaction and the content of an integrated interaction. The method 
further comprises the step of generating a notification message concerning 
misconduct in association with the at least one first interaction, with the at least 
one second interaction, or the integrated interaction. The method further 

5 comprises the step of collecting additional information. According to the 
invention the method further comprises the step of analyzing the collected 
additional information. The step of analyzing the transaction and the associated 
integrated interaction further . provides thereby analyzing the interaction 
associated witii a dispute. The step of analyzmg further provides for regulatory 

10 compliance. The step of analyzing further provides for quality management. The 
method further comprises the step of monitoring in real-time the content of the at 
least one first interaction, the at least one second interaction, and the integrated 
interaction. The step of monitoring further provides for the detection of 
malpractice. The method further comprises the. step of automatically scheduling 

15 and automatic selection of the integrated interaction to be monitored, analyzed 
and evaluated. ' ronizing of the at least one first interaction with the at 

r least one secoi • ruction is a time-based synchronizing process or is a 

sequential-based synchronizing process. The at least one first interaction and the 
at least one second interaction are identified by an interaction identifier. The 

20 interaction identifier is a unique point-in-time based identifier. The interaction 
identifier is a unique identifier. The at least one first interaction and the at least 
one second interaction constitute a time-based sequence of events. 

BRIEF DESCRIPTION OF THE DRAWINGS 
25 The present invention will be understood and appreciated more fiilly 

from the following detailed description taken in conjunction with the drawings in 
which: 

Fig. 1 is a schematic illustration of a multiple multimedia interaction 
recording and logging system, as known in the art; 
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Fig. 2 is a schematic iliiistration of a multiple multimedia interaction 
recording and logging system, in accordance with a preferred embodiment of the 
present invention; 

Fig. 3 is depicts the operative layers of the multiple multimedia 
5 interaction recording and logging system, in accordance with a preferred 
embodiment of the present invention; 

Fig. 4 is a schematic block diagram of the components constituting the 
multimedia interaction capture layer, in accordance with a preferred embodiment 
of the present invention; 
10 Fig. 5 A is a schematic illustration of the initial set-up and calibration 

layer, in accordance with another preferred embodiment of the present invention; 

3 Fig. 5B is a schematic illustration of the interaction archiving layer, 
in accordance with another preferred embodiment of the present invention; 

Fig. 6 is a schematic illustration of the additional information capture 
15 layer, in accordance with a further preferred embodiment of the present invention; 

Fig* 7A is a schematic illustration of the interaction access layer, in 
accordance with a preferred embodiment of the present invention; 

Fig. 7B is a schematic illustration of the association layer, in 
accordance with a preferred embodiment of the present invention; 
20 Fig. 7C is a schematic illustration of the seciirity layer, in accordance 

with a preferred embodiment of the present invention; 

Fig. 8A is a schematic illustration of the financial information layer, in 
accordance with a preferred embodiment of the present invention; 

Fig. 8B is a schematic illustration of the application layer, in 
25 accordance with a preferred embodiment of the present invention; 

Fig. 8C is a schematic illustration of the business layer, in accordance 
with a preferred embodiment of the present invention; and 

Fig. 9 is a schematic block diagram describing the operative 
components of the proposed system, in accordance with a preferred embodiment 
30 of the present invention. 
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DETAILED DESCRIPllON OF THE PREFERRED EMBODIMENT 

An apparatus, system and method for the recording and integrated 
archiving of multi-channel, multi-media and rich media interactions in financial 
institutions are disclosed. The proposed apparatus, system and method associates 

5 the interactions in a synchronized and integrative manner with the relevant 
business transactions in order to provide for integrative search, secured retrieval, 
ready playback and efficient review of the interactions and the proper time- 
sequenced re-constmction of an associated transaction process. In addition, the 
proposed apparatus, system, and method are provided with built-in capabilities 

10 that enable the generation of business analysis and performance information. The 
system fiirther includes specific, advanced tools for detecting suspicious 
behavioral patterns of users of the system, such as traders and supervisors. The 
proposed apparatus, system, and method fiirther provide quality management 
application to promote quality of service in financial institutions. Despite the 

15 diversity of needs and dissimilarities concerning the specific regulations to be 
enforced in different countries, the system is designed such as to co^; - : • 
-J essential needs while maintaining scalabilities in order to meet the 
changes that characterize the global financial market. 

The proposed system provides automatic scheduling and surveillance 

20 using real time content analysis, off-line analysis and key words spotting 
indicating either after work hour activity or "excessive Mendliness" expressions 
indicating giving or promismg gifts or benefits made by the customer, or direct 
use of words such as "presents", "gifts", "party", and the lilce. [ 

The above features could alert the management or could record it for 

25 fiiture surveillance monitoring the fi^equency of gifts received or provided. The 
proposed apparatus, system, and method are operative in the. integration of the 
content analysis framework with Computer Telephony Integration (CT£) and the 
Customer Relationship Management (CRM) systems in order to provide for a 
tracking capability. The tracking is accomplished via the utilization of business 
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analysis and detection attempts regarding the beneficiary. Suitable options for the 
notification of the management are disclosed as well. - 

The proposed apparatus, system and method provide tiie capability of 
capturing the screen activity of the employees' of the financial institution m order 

5 to verify and validate, the behavior of the employee along the sequence of the 
transaction execution. It will be evident that the present invention relates to 
mteractions rather than to complete transactions. It may be the case that any 
interaction is made for the purpose of inquiry or for receiving information for 
future potential transactions and the like. Irrespective of such intemctions tiie 

10 present invention is also directed towards interactions resulting in transactions. 
» Any indications of irregularity will trigger malpractice alerts to be communicated 
to the management. It utilizes diverse database sources, such as the organization 
knowledge base that contains access to customer's transactional information, 
additional customer's personal information such as social security mmiber, 

15 personal address, CTI information, such as source of call, caller ID, Automatic 
Number Identification (ANI), Dialed Niraiber ■ , '^^. ntion Service (DNIS), and 
/the like, and the content analysis data base J' / v^j, such as words spotted, 
accents detected, speaker verification results, and the like. The information is 
processed in order to detect unfamiliar customers and to trigger further 

20 investigation by the management. 

Content analysis tools are used in the context of the present invention and 
iiiherently supports multi-media inputs, correlation between the interaction of 
parties recorded and text extraction from the acknowledged information captured, 
confirming transaction details, such as price. Content analysis of both 

25 synchronized video and audio is performed to detect breaches of confidentiality 
occurring off-premises or outdoors. Video content analysis was described in 
detail in the referenced co-pending PCT applications: patent application serial 
number PCT/IL03/00097 titled METHOD AND APPARATUS FOR VIDEO 
FRAME SEQUENCE-BASED OBJECT TI^CKING filed 6 Febraaiy 2003 and 

30 walk-in centers face to face recording of video and audio was described in detail 

11 
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in the referenced co-pending PCX patent application serial niunber 
PCT/IL02/00741 titled RECORDING AND QULAITY MANAGEMENT FOR 
WALK-IN ENVIRONMENTS filed 5 September 2002 and content analysis on 
video and audio described in detail in US patent application serial number 
2002/01 10264B1 titled VIDEO AND AUDIO CONTENT ANALYSIS SYSTEM 
filed 15 August 2002. 



In addition, the proposed apparatus, system and method provides the option 
of investigating employees' interactive screen activity to detect the source of 

10 specious information via the capturing of web-co browsing activities, UftL 
selections, and the like, gathering information on practice pattem, alongside with 
the voice captured content analysis spotted words in helping to mold the 
employees' particular practices pattems. The most straightforward solution 
utilized for the detection of specious information involved is the detection of 

15 specific pre-deteraiined phrases like "I heard", ''rumor has it", and the like. Inter- 
institutional issues .iolved using the inherent capability of captured 
^interaction playback / - it? with the screen event activities generated/ The 
usage of content analysis can further generate cross processing of results to assist 
rapid dispute resolution. A plurality of business information is collected by the 

20 system of the present invention and is utilized through employing content analysis 
methods; such as speech analysis alongside with word or sentence spotting that is 
correlated with the employees' screen activity in order to detect irregularities and 
in order to alert the management to possible malpractices. Note should be taken 
that personal account monitoring is a delicate task since the employees' state of 

25 mind and private concems are potentially involved. Speech analysis is utilized to 
detect certain voice characteristics, such as stress, cross talk, stuttering, tone 
vibration, and the like. Speech analysis is fijrther operative in word or sentence 
spotting. The results of the analysis m conjunction with the captured records 
generated by the employees' screen activity could indicate irregularity in order to 

30 trigger alerts to the management in regard to a potential malpractice. 
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Referring now to Fig. 1 showing a typical multi sub-systems interaction 
recording and logging system 10 used by financial institutions. The system 10 
comprises several sub-systems where each sub-system is associated with a 
specific interaction type based on a specific communication method. Eacli sub- 

5 system is designed to implement an interaction of a specific type, to record and to 
store the record of the interaction and to provide suitable tools for the subsequent 
processing of the interaction records, such as interaction location, display, review, 
and tiie like. The system 10 includes an audio mteraction recordmg sub-system 
12, a fax interaction recording sub-system 14, an electronic mail (e-mail) 

10 interaction recording sub-system 16, and an instant messaging (SM) interaction- 
recording sub-system 18. The audio interaction sub-system 12 includes audio 
interactions 19, an audio interactions recorder device 20, audio mteractions 
archives 22, an audio interactions search component 24 and an audio interaction 
replay component 26. The fax interaction sub-system 14 includes fax mteractions 

15 27, a fax application 28, fax records archives 30, a fax records search component 
1 a fax records display component 34. The e-mail interaction sub-system 16 
I : interactions 35, an e-mail application 36, e-mail messages 

archives 40, an e-mail messages search component 40 and an e-mail messages 
display component 42. The IM interaction sub-system 18 includes IM interactions 

20 43, an IM application 44, IM messages archives 46, an IM messages search 
component 48 and an IM messages display component 50. It would be readily 
perceived that each of the recording sub-systems 12, 14, 16, 18 operate separately 
and independently in respect to each other. The separate sub-systems 12, 14, 16, 
18 are utilized for the implementation of financial transaction-specific 

25 interactions between two or more participants. The sub-systems 12, 14, 16, 18 are 
based on the utilization of specific communication channels via which the 
interactions are performed. Thus, the audio interactions sub-system 12 could 
utilize a voice communication channel constituting phone devices, telephone 
lines, line interface devices, telephone exchanges, and the like. The audio 

30 interactions recorder device 20 typically records the audio interactions 19. The 
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audio records representing fhe content of the voice interactions are stored in the 
audio interactions archives 22. The archives 22 are accessible via tiie audio 
interactions search method 24. The search 24 lypically provides the option of 
simple manual searches. The manual search could be conducted through a Call 

5 Management System server that has all the parameters of the calls. A person 
submits queries that include specific search parameters, such as for example, calls 
performed on a specific data from specific telephone, number to a Call 
Management System. The Call Management responds by providing the option of 
playback for all or a part of those calls that comply with the search parameters. 

10 The manual search could be fiirther performed through the interaction logger that 
includes a database storing the previously performed calls. Subsequent to the 
location of the records by the search method 24 the records are replayed by the 
audio interaction replay routine 26 to provide for interaction review, transaction 
process review and transaction process re-construction. In some cases a 

15 . transaction may comprise a smgle interaction type such as a single phone call 
resulting in a transaction. Incomplete transactions or other single interactior >: 
i^may be analyzed by the present system. In one example, a call made to • - 
information may be analyzed by the present invention. 

Still referring to Fig. 1 similarly the fax interactions sub-system 14, the e- 

20 mail interactions sub-system 16, and the IM interactions sub-system 18 utilize fax 
communication channels, e-mail communications channels, and IM messaging 
communications channels, respectively. The fax messages are handled by the fax 
application 28. Copies of the completed fax interactions 27 could be archived 
either manuedly or automatically into the fax records archives 30. The fax records 

25 search method 32 provides the option of locating specific fax interactions records 
stored m the archives 30. Locating such faxes can be performed according to 
simple parameters relating to the fax messages and recorded such as the fax 
sender or receiver, but not the content of the fax. Subsequent to the location of a 
specific fax interaction record in the archives 32 the record could be displayed by 
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fhe fax records display component 34 for the purposes of transaction process 
review and transaction process re-construction. 

Still referring to Fig. 1, e-mail interactions 35 typically comprise incoming 
and outgoing e-mail messages transmitted typically through a data 

5 communication network. The e-mail messages are handled by the e-mail 
application 36. Records reflecting the completed e-mail interactions 35 are 
typically archived into the e-mail messages archives 38. The search is typically 
performed by sequential browsing through the archives via the utilization of 
suitable software controls connected to the computer-based storage device. Such 

10 search is based on fields in the particular e-mails messages captured. Subsequent 
to the location of a specific e-mail message record in the archives 40 the record 
could be displayed by the e-mail messages display routine 42 for the purposes of 
interaction review, transaction process review and transaction process re- 
construction. 

15 Still referring to Fig. 1 IM interactions 43 typically comprise IM messages 

transmitted typically through a data communicatic a ; m lie. The EM messages 
are handled by the IM application 44. Recordf? ; : _ig the completed IM 
interactions 43 are typically archived into the IM messages archives 46. The IM 
messages search routine 48 provides the option of locating specific IM messaging 

20 records stored in the archives 46. The search is typically performed by sequential 
browsing through the archives via the utilization of suitable software controls 
connected to the computer-based storage device. Such search is based on fields in 
the particular IM messages captured. Subsequent to the location of a specific IM 
message record in the archives 46 the record could be displayed by the IM 

25 messaging display routine 50 for the purposes of interaction review, transaction 
process review and transaction process re-construction. 

Referring now to Fig. 2 showing a schematic illustration of a multiple 
multimedia interaction recording and logging system, in accordance with a 
preferred embodiment of the present invention. The recording and integrative 

30 logging system 50 includes media type interactions 51, 52, 54, 56, media type 
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capture units 58, 60, 62, 64, and an integrated logging platform 65. Media type 
interactions 51, 52, 54, 56 refer to diverse separate communication interfaces 
associated with diverse separate communication channels that provide interaction 
capabilities among the participants, such as, for example, a customer and a trader, 
5 for the initiation, progressive realization and final completion of a financial 
transaction between a customer and a financial institution or between two or more 
financial institutions. The media type, interactions 51, 52, 54, 56 involve 
exchanges of transaction-specific information, the introduction and submittal or 
transaction-specific instructions or orders, order-specific confirmations, 

10 corrections to previous • orders, negotiations, and the like. The media type 
interactions 51, 52, 54, 56 could involve an audio or video interaction, such as 
direct face-to-face voice conversation or a phone call, electronic exchange of 
financial transaction-specific data, such as e-mail messaging, instant messaging, 
fax messaging, captured video showing the persons conversing or exchanging 

15 information, and the like. Various other types of media type interactions could be 
used, including analog ar/J ^ multi-media and rich-media interactions. The 
different interactions 5i>, >4, 56 are performed in accordance to the 
configuration and pre-defined operating options associated with specific hardware 
and software interfaces. Thus, an audio interaction could be performed via a 

20 Computer Telephony Integration (CTI) mechanism in association with audio 
commimication devices, such as phone devices, communication lines and various 
suitable software modules. Similarly, a fax interaction is based on conventional 
fax machines, and conventional communication lines. An e-mail interaction could 
be performed by appropriate software routines constituting an e-mail application 

25 installed in a personal computer, where the submitted messages are transmitted 
between the participants via a data communication network, such as the Internet. 
Each interaction may be recorded by a video camera, via analog or digital 
recording device. The recorded file or analog recording is associated with the 
interaction recorded and transaction being performed. The media type capture 

30 units 58, 60, 62, 64 are linked to the media type interaction specific interfaces. 
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The capture units 58, 60, 62, 64 are operative in the recording of the content of 
the media type interactions where the manner of the recording is in accordance 
with the interaction interface. Thus, and audio hateraction generates audio records, 
a fax interaction, for example, generates text-based fax records, an e-mail 
5 interaction generates rich-text based e-mail message, a video interaction generates 
video frames, and the like. The media type capture devices may include a 
telephone, a fax, an e-mail server, an analog or digital video camera, an analog or 
digital audio microphone and other devices capable of capturing interactions 
between the parties to a transaction. Analog captured information is preferably 
10 converted into digital form to enable streamlining of the information handled by 
the system of the present invention, though non-transformed analog captured 
media may also be managed through the use of records related and pointing to the 
analog media serving as the connectors to the analog media. 

Still referring to Fig. 2 the separately performed and captured media type 
15 interaction content records are transferred to the integrated logging platform 65. 
17: ^ -."rm 65 is typically a computing device wiA multi-media processing and 
w ; capabilities. The platform 65 further provides specific record data 
retrieval and playback features. The platform 65 receives the separately generated 
and captured interaction records, and processes the records in order to provide for 
20 interaction integration 66, for time synchronization 68 and for the association 70 
of the interaction records to a relevant financial transaction or transactions. The 
integrated, time-synchronized and transaction-associated interaction records are 
stored in an integrated database 72 as multi-media data records, encoded into 
machine-readable format where each type of record is structured in accordance 
25 with type of the media represented by the record. Thus, the database 72 could 
include audio records, text records, video records, scanned records, data records, 
and the like. The database 72 further includes pre-defined metadata information 
concerning the media type of the potential records, the location, structure and 
organization of the records, the indexing schemas, and the like. The database 72 
30 could include raw data records representing the interactions or pointer records 
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that indicate storage location of the interaction records. The integrated collection 
of the multi-media records provide the option for flie integrated retrieval 76 of 
one or more interaction record and for the integrated playback 74 of one or more 
interaction record for purposes of interaction review, and transaction process re- 
5 construction. Thus, . the integrated retrieval 76 and the associated integrated 
playback 74 provide for enhanced regulation compliance, rapid dispute 
resolution, improved quality management, more efficient methods for the 
detection of suspicious behavior patterns, business patterns analysis, and the like. 
The features of the integrated logging platform 65 provide for substantially 

10 improved records retrieval, playback, and review methods in respect of the 
currently offered systems. Note should be taken than on the drawing under 
discussion a limited number of media type interactions and a limited type of 
associated media type capture units are shown. Furthermore, on the drawing 
imder discussion only a single integrated logging platform is shown. It would be 

15 readily perceived that in a realistic environment a plurality of media type 
interactions could generate interaction records, which could be captured b3^ - 
plurality of media type capturing units. In practice, the plurality of the interactl^ii: 
records could be transmitted and stored on more than one integrated logging 
platforms in accordance with a predefined system configuration. Note should be 

20 taken that the transactions generated following the performance of the 
interactions could be completed interactions or non-completed transactions. 
Referring now to Fig.3 that depicts the operative layers of the multiple 
multimedia interaction recording and logging system, in accordance with a 
preferred embodiment of the present invention. The multi-channel multimedia 

25 capturing, logging and retrieval apparatus and system 80 provides time- 
synchronized voice and data interactions capturing, logging and secure playback 
and retrieval capabilities for quality management, for dispute resolution, for 
positive and negative conduct detection, and for business analysis, and financial 
institutions. The apparatus 80 is build firom layers, and components, devices and 

30 methods embedded in specific routines, where each layer, component, device and 

18 



wo 2004/090770 i^^CT/IL2003/000300 

imbedded method can be implemented eiiher as described in association with the 
drawing or can be implemented as part of a different layer. The layers, 
components, devices and methods embedded in specific routines can be deployed 
on a single platform, could be distributed locally across a Local Area Network 
5 (LAN), could be distributed remotely across a Wide Area Network (WAN), or the 
like. The supporting networks could be IP networks, wireless networks, satellite 
networks, radio networks, cellular networks, CATV distribution networks, and 
the like. For example the media type capture units 58, 60, 62, 64 of Fig. 2 could 
be located in the front office desks in a New York branch of a financial institution 

10 while the integrated logging platform 65 of Fig. 2 could be deployed in the back 
office site located in Washington D.C. where the remote devices communicate via 
a Wide Area Network (WAN). The system 80 includes a multi-media interaction 
layer 81, an initial set-up and calibration layer 82, an interaction archives layer 
84, an additional information layer 86, an interaction access layer 88, an 

15 association layer 90, a security layer 92, a financial information systems layer 94, 
an application layer 96, and a business layer 98. The muf * :\ ' s interaction layer 
81 includes the various interaction types that conj i esent in typical ^ 

financial institutions, such as instant messaging, e-mail, chats, web co-browsing, 
phone calls, screen sequence evens, video, audio, survey results, and the like. 

20 Such interactions take place between two or more participants that are parties in 
the initiation, progressive negotiation, and completion of a financial transaction. 
The participants could include diverse employees or representatives of one or 
more financial institutions as well as private customers. The employees typically 
include traders, brokers, back office representatives, sales representatives, other 

25 financial institutes, trading floor representatives, and the like. The customers 
typically include private customers, institutional customers, and the like. The 
fimctionality of the layer 81 is to provide interaction-specific multi-media records 
where the content of tiie records reflect the substance of the associated 
interactions. The initial set-up and calibration layer 82 is responsible for the 

30 configuration of the system and for the setting up of the system, in accordance 
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with the multi media interactions types found on the particular financial institute 
site. The configuration/setup/calibration could be accomplished on site in order to 
make sure that the functionality of the system corresponds with the requirements 
of the customer and the environment. During the lifetime of the system, 

S calibration can be p^ormed adaptively in accordance with the institution's 
profile and accumulated changes. The interaction archives layer 84 provides for 
the archiving of the interaction records and includes the various relevant 
databases. The additional information layer 86 are being used with the interaction 
archives information for the purpose of gathering real-time information on the 

10 content. of the information in order to monitor the conipliance with institutional 
regulations, pre-defined code of conduct and other local organization rules, State 
and Federal or industry wide rules with respect to the interactions. The interaction 
access layer 88 locates and retrieves interaction records fi-om the interactions 
archives while using metadata information and the additional sources of 

IS information capture layer 86. The association layer 90 is designed to associate the 
• transaction in the financial H- !3 Iransaction system with the previous 
interactions that led up to fi: kept in the interaction archiving layer 

84. The security layer 92 is responsible for complying with the code of conduct of 
financial institutes, which demands that access for playback and retrieval of 

20 interactions firom the interaction archives will be managed in a controllable and 
restricted manner. The fibnancial information systems layer 94 is the central 
system in the organization. The financial information system 94 comprises an 
existing transactional system of the organization. The system 94 is used for the 
entering and storage of transactional information, such as closing tickets, trade 

25 information, completed transactions information, and the like. The system 94 
could be a CRM system. Presently, such a system operates typically 
independently of the logging devices. _The system 94 is utilized as the main 
access point for the trader. In accordance with the preferred embodiments of the 
present invention, the system 94 uses integrated components in order to provide 

30 the ability to perform the unique functions provided by the proposed system and 
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method. Hie proposed system and method operates preferably through the system 
94 or operates in conjmiction with tiie system 94... The application layer 96 
provides diverse applications with added value to the financial institutes. These 
applications could include reportmg, statistics tools and applications, e-leaming 

5 applications designed to improve the performance quality of the financial 
institute's representatives, and the like. The business layer 98 is designed to 
provide high-level analysis of the integrated transactional interaction metadata, 
for the identification of trends and for formulating business intelligence 
information. Through the utilization of the business layer 98 managers, 

10 compliance officers and auditors can receive insight regarding improvements in 
service, successful and unsuccessful financial products, market trends, detection 
of proper and improper conduct, inspection and promotion, regulation compliance 
assessment, drill down analysis, business performance information and the like. 
Referring now to Fig. At showing a schematic block diagram of the 

15 components constituting the multimedia interaction capture layer. The multi- 
media?, -ons layer includes non-limiting examples of various interaction 
r type.v .. ixionly present in financial institutes. The various interaction types 
were discussed herein above and could include media types, such as instant 
messaging 108, e-mail 110, chat 112, web co-browsing 114, survey results 116, 

20 which for example might proceed a phone call, screen sequence events 118 
captured from within the desktop or other devices of a participant of the 
interaction, video images 104, and audio information 106 such as Voice over IP 
(VoIP), and the like. 

Referring now to Fig. 5 A the set-up and calibration layer 122 includes an 

25 on-site business oriented set-up module 124 and an adaptive set-up module 126. 
The on-site business oriented set-up module 124 provides for setting, up the 
system in accordance with the needs of the financial institution as mandated by 
the operator of the system. One example demonstrating the functionality of the 
set-up module 124 concems generating and setting certain parameters of a word- 

30 spotting engine. In trading floors a set of common words, such as "buy", "shares", 
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"call option'', "trader" and the like, would be introduced during a setup in order to 
enhance the results of a word spotting engine.. Similarly words such as "gift", 
*T5ef' , "alcohol" and other like words may be screened for. Another example of 
set-up information relates to the conflict of interest that arises if traders are 

5 permitted to deal for themselves in those commodities, instruments or products 
related to the ones in which they deal for their institution. In case dealing for 
personal account is permitted management should be able control and monitor 
and detect abuse cases such as "front-running", where an employee could execute 
a personal trade in advance of a client's or institutional order to benefit from an 

10 anticipated movement in the market The trader's screen activity 118 of Fig. 4 in 
conjunction with the order for execution of the trade, are captured such that 
behavior of the employee is verified along the sequence of execution. The system 
will alert the management using an alert module (not shown) when an indication 
of suspected irregularity or possible malpractice is identified. In addition, during 

15 the lifetime of the system, the calibration and the set-up could be performed 
adaptively via the utilization of the adaptive set-up module 126 in accordance 
with profile and requirements of the institution and dynamically accumulated 
changes. 

Referring to Fig. 5B the interaction archives layer 128 includes an 
20 interaction raw data database 130, an interaction metadata database 132, a 
knowledge database 134, and additional organization databases 136, such as 
CRM DB, ERP DB, Knowledge Base, and the like. The interaction raw data 
database 130 contains the actual interaction contents recorded and retained. The 
interaction metadata database 132 contains parameters and attributes associated 
25 with the raw data included in the interaction. Non-limiting examples include CTI 
information, retention and migration parameters, and the like. For example, the 
retention parameters concern the length of the period during which the call will be 
retamed in the raw data base 130, such as long term retention or short term 
retention, before it will be erased and becomes non retrievable. The database 132 
30 further contains migration parameters that define the destination media storage, 
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such as DVD3 DAT, Jukebox, Storage Area Network (SAN), designated file 
server, and the like, to which the interaction will be routed, stored into, and 
maintained on. 

Referring now to Fig. 6 the additional data sources of the additional 

5 information layer 140 are being used with the interaction archives information for 
the purpose of collecting real-time information on the content of the interactions 
in order to monitor the code of conduct. Call information is received either from 
the Automatic Call Distributor (ACD) or from the ACD through the CIT. Call 
information coming from the ACD can be used in monitoring trader activity in 

10 real-time during the engagement of the trader in an interaction with a customer. 
Call mformation can also arrive from a Turret system, also known as a Dealer 
Board or from a Private Branch Exchange (PBX) system. CTI inforaiation 142 is 
of import since it is substantially accurate and could provide information about 
the length of the call, the calling number and the called number and extension 

15 through the Dial Number Identification Service, the agent, and customer IDs, the 
customer classification, and the like. Once the system :*seived CTI 

information 142 it is provided with a better capability ... pre-process 
interactions and later performing rule-based analysis 144 leading to enhanced 
results. CTI information 142 is utilized in conjimction with other transactional 

20 information, such as CRM data 146 and Enterprise Resource Planning (ERP) data 
148, as additional inputs to be available by the application layer and during ojff- 
line business analysis. Content analysis information 144 was described in detail in 
the referenced co-pending PCT patent application number PCT/IL02/00593 titled 
Method, Apparatus and System for Capturing and Analyzing Interaction Based 

25 Content, filed 18 July 2002. The time synchronization module 150 is designed to 
assist in the monitoring of the correct temporal sequence of events that occurred 
during the complete interaction in order to assure correct scenario reconstruction 
of events, for example for dispute resolution purposes. These purposes are 
achieved by synchronizing the timing devices of the servers and devices that 

30 handle interaction and transactional information. In order to identify money 
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laundering and terror funding issues the additional information layer 140 provides 
solution for customer identification and verification. Customer identification 
could be achieved tiirough various algorithms that the layer 140 could 
incorporate. One such algorithm for customw identification, for example, could 

5 be the Comverse PerSay solution. An alternative solution for customer 
identification that could be incorporated in the layer 140 is described in the 
related US patent application serial number 2002/01 10264B1 titled VIDEO AND 
AUDIO CONTENT ANALYSIS SYSTEM. 15 filed August 2002. 

Referring now to Fig. 7 A the mteraction access layer 152 locates and 

10 retrieves interactions by using the metadata information 132 of the interactions 
archives layer 84, in conjxmction with the additional information sources firom the 
additional information layer 140. Requests for interaction retrieval from the 
application layer 96 or firom the financial information systems layer 94 are sent to 
the integrated retrieval module 154 and the integrated playback module 156. The 

15 module 154 retrieves the required interactions and allows tiie reconstruction of 
events and interactions that coiVr . r r>^fs complete transaction flow. The 
mnovative part of this module is f& ^ -aced approach for presenting a single 
view of all types of interactions reconstructed in a synchronized order with 
accordance to the original generation of the interaction records. 

20 Referring now to Fig. 7B the association layer 158 includes the 

transaction to interaction association module 160. The association layer 158 is 
responsible for associating a transaction in the financial institute's transaction 
system with the transaction-specific set of interactions that led to the transaction. 
The transaction-specific set of interactions is stored in the interaction archives of 

25 the interaction archives layer. The association of the transaction with the relevant 
set of transactions is accomplished by the utilization of an integration component. 
The innovation in the transaction to interaction association module 160 is evident 
firom the manner the module 160 uniquely identifies each transaction and by 
using the integration component enable to link the set of interactions with 

30 relevant transactional data in the transaction system. The integration component 
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and the transaction system will be described in more detail herein under in 
association with the following drawings. 

Referring now to Fig. 7C the security layer 162 is responsible for 
complying with the code of conduct of financial institutes, which demands that 
5 access for the playback and the retrieval of interactions from the interaction 
archives will be managed in a secure, controllable and restricted manner. The 
secure access interface 164 is responsible for granting access to authorized 
personnel upon request The interface 164 could be schedule-based via the 
utilization of an interaction-based security mechanism, which can be set up with 
10 or without time limitations. 

Fig. 8A describes the financial information systems layer 168. The 
layer 168 includes the financial institute's transaction system 170. The system 170 
includes the financial transactions generated following one or more associated 
interactions. The system 170 is the central system associated with the institution. 
15 The system 170 is utilized for the entering of activity information by a trader. 
Such infDi^ ould include instructions provided to the trader from a client, 

^ where eacl action could include transactional amount, name of the 
participants in the transaction, beneficiaries, transaction type, and the like. One 
example of such a system is the Siebel CRM system designed and developed for 
20 financial institutions. The layer 168 fiirther includes the integration component 
172. The component 172 is responsible for associating a transaction stored in the 
transaction system 170 with a set of one or more interactions that led up to the 
generation of the transaction. The component 172 in conjunction with the 
transaction to interaction association module 160 of Fig. 7B links the relevant set 
25 of interactions with the suitable transactional data in the transaction system 170. 
The integration component 172 can be activated from within the fijiancial 
institute's transactional system 170 providing a transparent retrieval and playback 
capabiUty. The integration component 172 could be an ActiveX software 
component embedded as part of the transactional system 170 and displayed as an 
30 icon on the graphical user interface of a computing device operated by a 
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representative of the financial institution in order to provide instant playback of 
the complete transaction-specific set of interactions- The integration component 
172 is substantially sknilar to the 'TSTice Link" product of Nice Systems Ltd 
certified for integration with Siebel CRM system. 
5 Referring now to Fig. 8B the application layer 174 includes a 

monitoring application 176, an e-leaming application 178, a reporting and 
statistics application 180, an evaluation application 182, a surveillance application 
184, and an automatic scheduler 186. The functionality of the application layer 
174 is the provision of diverse value-added applications to the financial 

10 institutions. The applications 176, 178, 180, 182, 184, 186 are designed to 
monitor conduct, to provide business performance infotmation and to improve the 
performance quality of the financial institute's representatives. Monitoring 
application 176 allows key personnel of the financial institution, such as 
compliance officers, management, auditors, and supervisors to monitor 

15 selectively one or more interactions in real time. Evaluations application 182 
allow managers of the financial institution to evaluate and assess the performance 
of the financial institute's representatives providing scores and deducing the skill 
of a representative using information collected in the additional information layer 
86 and the interaction archives layer 84. The application 182 plays back the 

20 multimedia interactions within the multi-media interactions layer 81 with the 
additional information firom the additional mformation layer 86 in order to 
improve quality and performance and to monitor that the behavior of the 
representatives is in compliance with the code of conduct or other rales. The 
automatic scheduler application 186 provides an automated selection of 

25 interactions generated and collected by the system for off-line evaluations in 
accordance with predefined rales. Through the use of the scheduler application 
186, a program can be scheduled to prepare, in a randomized manner, for 
example, thirty calls firom the last twenty-four hours for a specific compliance 
officer in order to provide the option to review the interactions through playing 

30 back and evaluating the content of the calls. Surveillance application 184 allow 
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compliance groups and managers to implement a more detailed surveillance on 
specific representatives, specific customers or third parties for the purpose of 
identifying any suspicious behavior or interaction which may stand in 
contradiction to the required code of conduct. For example, the management 

5 should control, monitor and detect cases such as "front running", where an agent 
could execute a personal trade in advance of a client's order or an institutional 
order, to benefit from an anticipated movement in the market. The surveillance 
application 184 is fiirther capable of setting a real-time alarm and/or generate and 
transmit a notification message where the system detects situations of 

10 misconduct. Such alarms or notifications provide the option for real time 
intervention during the performance of the interactions. The application 184 
further provides an alarm indication or a notification to a compliance officer in 
cases where a dispute remains open or unresolved for a predefined period of time. 
A specific period of time could be defined during the activation of the system set 

15 up via the setup layer 82 or could be defined subsequently (preferably by a 
compliance officer) in order to handle a particular case. Referring : --ig. 8C 

V the business layer 188 is designed to provide high level analysis vlegrated 
transactional interaction metadata, for the identification of trends and for 
formulating business intelligence information. Through the utilization of the 

20 business layer 188 managers, compliance officers and auditors can receive insight 
regarding improvements in service, successful and unsuccessful financial 
products, market trends, detection of proper and improper conduct, inspection and 
promotion, regulation compliance assessment, drill down analysis, business 
performance and tiie like Analysis results are stored in the organization's 

25 knowledge base database 134 in the interaction archives layer 128 of Fig. 5B. The 
stored results can further be used for adaptive purposes such as modeling and 
identifying changes in behavior, and feed tiie results back into the system. The 
feedback can be used to place automatically suspected parties such as traders or 
even the compliance officers themselves xmder system surveillance due to 

30 suspicious indicators previously detected. A non-limiting example concerns a 
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senior trader fhat just started to use drugs where the organization is an aware of 
this harmful situation. The trader was previously known to the organization as a 
highly competent, skilled, and hard worker who adequately follows regulations of 
the organization. His personal revenues, dealing with the organization 

5 commodities, gained constant growth in past 12 months. The system stores the 
profile of the trader and records all the transactional achievements and the 
interactions handled. At some point in time several changes are identified in the 
hehavior of the trader. The system detects monitors and records the changes by 
utilizing information firom the additional information layer 140 of Fig. 6. The 

10 content analysis engines detect high tones and aggravated behavioral stress during 
several interactions. The number of cross talk phenomena dramatically increases. 
The trader submits repeated requests for playback of the same interaction and his 
captured screen events indicate an increased number of backspace keystrokes and 
multiple deletions during the interaction sessions. These suspicious behavioral 

15 patterns are not in accordance with the trader's past profile. Therefore, the system 
marks the trader for surveillance. In ext; i . . don were considerable amounts 
of money are involved and a suspicious : al pattern is detected, a real tinie 

alarm is triggered and/or a notification routine is activated in order to transmit a 
suitable message calling for supervisor intervention. The supervisor then will act 

20 in the appropriate manner to prevent the trader for exposing tiie organization to 
risk. 

The code of conduct in financial institutions requires that there will be 
a complete view of all the customer's interactions with the financial institution. 
One of the problems that need to be resolved is that a third party can interact with 

25 the financial institute using, for example, an email address ^ W, using an Instant 
Messaging account ''B", dial-in from a phone mraiber "C", and the like. The 
financial institute's representative can also use different identifiers during 
interactions, such as e-mail address, IM accoimt number, phone extension 
number, and the like. Another integration obstacle is the time synchronization 

30 between the transactional and the logging systems. The code of conduct requires 
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that all the interactions will be displayed in the temporal sequence . in which they 
occurred, since an instruction given in later interaction should override any 
instruction given in previous interactions. 

Referring now to Fig. 9 showing, a schematic block diagram 
5 describing the components of the system, in accordance with a preferred 
embodiment of the present invention. Requests for interaction retrieval and 
playback can origmate from either the application layer (not shown), from the 
integrated retrieval and playback 208 or preferably directly from the transaction 
system 222. The requests could include submitting a request for playback, 
10 submitting a query on certain interactions, and the like. The ability to embedded 
query, retrieval and playback capabilities (that inherently belongs to the logging 
system) into a different system such as the transaction system 222 is done by 
using the integration component 224. The component 224 is operative in the 
linking of the transaction system with the mteraction archiving systems. The link 
15 could be accomplished via the utilization of ActiveX components that maintain a 
reference poii?t^ . interactions tables located in the archives 196 associating 
: the interactic:. .^ord metadata with the transactional information in the 

transaction system 222. The request is then sent to the integrated 
retrieval/playback module 208, which is used to provide an integrated view of ail 
20 the uiteractions captured m the financial institute. The integrated 
retrieval/playback module 208 is capable of locating all multimedia interactions 
that constitute the single transaction required to be retrieved. The mtegrated 
retrieval and playback component 208 uses the sources in the additional 
information sources 220, such as the real-time CTI information 216, the content 
25 analysis results 218, and the collected meta-data 198, 200 of the interactions 
archives 196 in conjunction with other organizational information, such as 
profile, accoimt numbers, social security number and the like, in order to make 
the association of all the different identifiers, such as e-mail account, instant 
messaging user id, phone extension number, account number, customer number, 
30 and the like. If, for example, there is a need to track a certain deal or sale that was 
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made by a specific financial institute's representative, then the integrated retrieval 
and playback module 208 will be used. The module 208 will conmiunicate with 
the meta-data database 198, 200, CRM information 221, ERP information 223, 
content analysis information 218, and CTI information 216 in order to identify 

5 correctly and track suitably the data and voice interactions which were made by 
the representative of the financial institution in accordance with diverse assigned 
identities supporting any of the multimedia type of interactions. 

The second fimbtion of the integrated retrieval/playback module 208 
concerns the reconstruction of the original temporal sequence in which tiie 

10 interactions occurred. The time synchronization module 212 is used for 
synchronizing the timing devices of the modules and devices that handled the 
transaction-specific interactions alongside other , sources of information systems, 
such as CRM system server, ERP system server. Content Analysis Server, CTI 
server, and the like. The module 212 constantly monitors and verifies that all the 

15 severs that communicate across the system are time synchronized, even if they are 
located remotely in respect to each other, since the timestamp generated by the 
timing devices should reflect the precise point in time associated with the 
occurrence of the interactions. An exemplary protocol for suitably synchronizing 
devices is the Network Time Protocol (NTP). Synchronization is used in order to 

20 reconstruct the correct temporal sequence of the interactions. The time 
synchronization of the timing devices of the interaction archives guarantees that 
the correct sequence of events is maintained, so the integrated retrieval/playback 
module 208 associates the transaction-specific set of interactions in accordance 
with to the timestamp generated at that point in time in which the interactions 

25 were captured. Transactions often require special investigation, either due to a 
dispute with a customer or due to a request fi:om an auditing body. The 
investigation will include the examination of the interactions that led to this 
interaction. For example, if a customer complains that an order he gave in a 
phone call was not executed correctly, the financial institute will search for the 

30 audio of the call and will playback the audio to determine whether the customer 
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was light or wrong. If an auditor would like to further investigate a suspicious 
looking transaction,, so the jQnancial institute will need to search for the 
interactions that led to this transaction to see the all picture. A suitable match 
between the actual transactions record to one or more interactions that led to that 
5 transaction in dispute is needed- The assumption behind this association and/or 
matching solution is that an interaction will always precede the transaction so a 
matching of a transaction to the interactions will be performed for each 
transaction as it is entered into the transaction system. A unique identifier will be 
created for each interaction and the identifier will then be entered into the 
10 transaction system alongside the details of the transaction. 

Still referring to Fig. 9 the process begins with the assignment of a 
unique identifier to each of the multirmedia interactions 192. The interaction 
identifier could be created by the capture unit of the interaction (not shown), such 
as the telephone switch in case of a telephone call, an exchange iserver in case of 
15 e-mail, or the voice and data metadata servers of the interaction archives layer. 
For example, an SQL database server could create the identifier and f+r: % m -dth 
as part of the metadata. The transaction to interaction association mo4 v : vvill 
receive the identities of the interactions in real time during the occurrence of the 
interaction. The module 206 will continue the matching process by presenting the 
20 interaction identifiers to the financial institute's representatives where each 
representative will be presented with the identifiers of his own interactions. The 
identifiers are presented to the representative by the utilization of the integration 
component 224. The component 224 runs on the computers of each representative 
either as a taskbar or as a background application. The representative submits the 
25 interaction identifiers to the component 224 which in tum will display the 
identification of the last interaction made by the representative. The application 
will also show the most recent identifiers along with details describing the 
interactions such as interaction id, time of interaction, media type, such as phone, 
email, instant message, and the like, third party identity and other such 
30 information. 
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Once an interaction has ended in a transaction, the representatives 
enter the details of the transaction into the iSnancial institute's transaction system 
222, To make the link between the transaction and the interactions that led to that 
transaction, the representatives would now need to manually enter the interaction 
5 identifier, which is displayed by the integration component 224, into, the 
transaction system 222. Currently, records of the transaction are recorded 
manually by the trader. First, typically a paper document, such as a pre-defined 
form, IS utilized by the trader in order to record the details of the transaction. 
Next, the trader will manually copy the details of the transaction firom the 
10 document into the transaction system 222. Presently, the details do not include the 
interaction type or the interaction content that lead to the transaction. 

In the preferred embodiment of the present invention the integration 
component 224 collects the interaction identifiers of each interaction type 
associated with each of the parties involved, , such as beneficiaries, trader, name of 
15 institutions involved in the transaction and the like. The integration components 
further collects the performed transactions. TJ! ' - - ^ .?^:atipn component 224 stores 
the collected information. The collected .:Ion could be retrieved, 

displayed via the transaction system 222 or via a separate component designed to 
display the information in a suitable manner. The integration component is linked 
20 to the other components of the system of the present invention and could, for 
example, provide a notification if there is a discrepancy between the actions of 
the trader vis-^-vis the transaction system 222 and the mteractions captured. An 
exemplary situation involving a discrepancy could arise where a transaction is 
identified through the analysis of the interaction using the content analysis engine 
25 although the trader failed to record the transaction, or failed to enter specific 
details, or entered the inaccurate details of the transaction into the transaction 
system 222. This will allow providmg proper notifications at the time transactions 
are entered mto the transaction system 222. Preferably an automatic process will 
be implemented where the integration component 224 will interact with the 
30 screen sequence events capturing mechanism 118 of Fig. 4. Screen events 
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capturing was described in detail in the referenced c9-pending US published 
patent appUcation number PCT/ELO 1/00796 titled SYSTEM AND METHOD 
FOR CAPTURING BROWSER SESION AND USER ACTION filed 24 August 
2001. Consequent to the entering of the transaction identifier into the suitable data 
5 field in the transactional system 222 all identifiers of the transaction-specific 
interaction types are associated via triggered association ("on-the-fly") by the 
transaction to interaction association module 206. Through the set-up and 
calibration 194 the tunmg of the system during setiq) can reflect the number of 
multimedia interactions and the buffer size that will hold the interaction 
10 identifiers, needed for the association process. The integration compoiient 224 
will examine the screen sequence of events. When a representative of the 
financial institution begins the process of entering a transaction into the 
transaction system 222 the integration component 224 will be triggered to mteract 
with the transaction system 222 in order to place automatically the interaction 
15 identifiers in the desired data fields either a specific transaction entry form or the 
screen. The identifi& - -isociated at database level 196 via the utilization of 
the transaction to x ^ Ion association module 206, for fiiture retrieved of 
complete transaction-specific interactions 

The financial institution archives, such as the interaction types 
20 databases, the transactional database or any additional information system, 
contain private and sensitive information. The code of conduct for financial 
institutes requires that access to such data should be limited and restricted 
especially in the case of the interaction archives. The requirement for managing 
and controlled access is typically characterizes financial organizations where 
25 access needed to be dynamically granted on a daily, per case basis, and where the 
organization is constantiy being monitored by representatives assigned by the 
institution's management, SEC, FSA, NASD or the like in order to detect 
misconduct. The proposed apparatus, system and method enables allow 
compliance officers or other authorized personnel to grant access to a 
30 representative requesting to retrieve and playback a specific interaction. The 
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representative will not be able to access any interactions other than those that 
were specifically granted to him, nor will he be able to make any modifications, to 
the original interaction, such as the raw data, and to the metadata. The secured 
access 214 guarantees data integrity that no one can interfere with the interaction 

5 types stored thereby assuring that the transaction-specific uiteractions are kept * 
secured, intact and are admissible in court. 

In addition, the interaction identifiers will be logged and linked with 
the record that describes the set of interactions. The interactions record could hold 
CTI real time parameters, such as caller ID, e-mail address, content analysis 

10 customer identification, and other identifiers, such as ANI number and DNIS 
number. Thus, during an interaction while entering the details of the transaction, 
the trader participating in the interaction could retrieve and playback the entire set 
of interactions since the interaction identifiers are linked together at database 
level. After the termination of the phone call the trader receives an e-mail from 

15 the same customer. The e-mail address of the customer is captured by the screen 
and received by the software driver connected to the e-mail server. Since 
same customer is involved with the same trader the phone call and the e-mail 
are linked and associated at the database level to the same transaction. When the 
trader enters the transaction information into the transaction system the system 

20 automatically associates tiie transaction with the set of interactions. 

* Still referring to Fig. 9 the secure access interface 214 will be the 
access permission gateway for all the above described applications 176, 178, 180, 
182, 184, 186 of the application layer 174 of Fig. 8B, All the applications that 
requires querying, locating, retrieving or playback one or more transaction- 

25 specific interactions required to obtain secure access interface 214 authorization. 
A trader, via for example tiie transaction system 222, submits a request for 
retrieval of one or more interactions previously performed. The trader desires to 
verify that he had followed the entire customer request and the purchasing of the 
right amount of shares was done in the proper manner. The security access 

30 interface 214 will decide whether this request should be granted or rejected based 
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on that particular trader profile set through the set-up and calibration 194 by the 
system administrator. As all the organization members' profiles, privileges, such 
as read, write, update delete, retention setting, and the like, access rights are kept 
and managed by an administrative database authorized administrators can set 

5 rules that resides in the secure access interface 214. These rules can be retrieved 
or accessed from the organization database upon request. When such a request is 
submitted the system automatically approves or rejects the request and display a 
suitable message with an attached short explanation. System administrators can 
be compliance officers or any other authorized personnel responsible for 

10 controlling the access to the archives. The administrator will also have an 
application interface (not shown) to the secure access interface 214 in order to 
approve or eject the requests for retrieval/playback. The secure access interface 
214 then generates a password, which will grant access only to the requested 
interactions. As the password is currently entered via the phone keyboard it is 

15 convenient to use a numeric password. The numeric password could be generated 
in a random manner or could be based on the Personal Identification N • - » 
(PIN) of the requestor. The system could be set-up 194 to use generated :> 
or PIN-based passwords, which are only valid for a time-limited period, such as a 
week, for example. Upon receiving the password, tiie requestor will be able to use 

20 the password passed to the integrated retrieval/playback 208 and to access 
required interactions in the archives 196. The system can be set-up in such a 
maimer as to provide further protection of the archives and for the prevention of 
direct access of the application to the archives by making a copy of the 
interactions stored in the archives 196 to the integrated retrieval/playback 208. 

25 This will ensure that even if the user will somehow override the system and find a 
way to bypass the secure access interface 214 and "manipulate" the interactions, 
it will not affect the original copy of the interaction, which stays intact and 
untouched in the genuine archives. 



30 interaction access. The first is better suited for data access and the later for voice 



There are two different ways for using this password protected 
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access. Using the &st method the requestor uses a computer-based application 
that accesses the secure access interface 214. The requestor then is enabled to 
view aU the requests he made and their status, and can see which requests have 
been approved. By selecting the request he wants to pursue he is then prompted to 

5 enter the access password. After validating that the password is correct and valid, 
the secure access interface 214 will command the integrated retrieval/ playback 
208 to transfer a copy of these interactions to the requestor's screen. Usmg the 
second method, in case audio interactions, it will be more convenient for the 
requestor to use his telephone handset for listening to a specific interaction. The 

io secure access interface 214 will support a dial-in based, accessed using, for 
example, an IVR server (not shown). The requester will be prompted by the 
system to enter his password, which could be entered by using the telephone 
digits or keys (0-9, *, #) and key in the password granted to him. If the password 
is correct and valid the system will access the integrated retrieval/playback 208 

15 and play the voice interactions to the requestor by phone. For transferring over 
the phone all types of voice interactions such as pJr Voice over IP, voice 

instant messages, and the like, the system can u ?.or art solutions such as 

"Executive (remote) connect" product of Nice System Ltd where remote access 
via Dual Tone Multi Frequency (DTMF) to a designated extension in a telephony 

20 switch, provides monitoring and pilayback of interactions captured by the logging 
system. In addition to voice interactions over the phone, it is also possible to 
translate other data interactions such as e-mails, instant messages, and the like, to 
voice, which will be transmitted over the phone. The system could use text-to- 
speech solutions in order to convert the text into voice and to "read" the text to 

25 the requester. The benefits of the executive connect second method is that it 
allows accessing the system even at times which the requestor is outside the 
office and only can access the organization by telephone. 

An important part of the work of compliance officers, auditors and 
managers in a financial institute is to sample the interactions of the representative 

30 with customers or with other interested parties in order to verify that the code of 
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conduct is followed. By reviewing a representative sample of the int^actions 
these persons, usually assign by the financial institution management, might 
locate, and identify cases of misconduct 

The proposed apparatus, system and method further . provide an 

5 automatic scheduler application 186 of Fig. 8B. The application 186 is configured 
according to a pre-defined setup performed via the on-site business oriented set- 
up 124 of Fig. 5 A that allows the business to determine which key personnel will 
be authorized to review the one or more transaction-specific interactions, the 
number of interactions to be reviewed during a specific period, such as during a 

10 week, a month, and the like. The setup 124 fiirfher allows the business to 
determine the business filtrating mles operative in deciding which interactions are 
more "interesting" for review. The automatic scheduler 186 will communicate 
with the voice archives, data archives, metadata archives, additional archives, and 
with the sources of the additional information layer 140 of Fig, 6, such as CTI 

15 information 142, CRM information 146, ERP information 148, and the like, in 
order to determine whi^:; vr^ /^^^ should be reviewed according to the 
^ business needs. More spr a scheduling program could be set up via the 

automatic scheduler application 186, or the system could automatically prepare 
according to the mitial setup and calibration business rules and action setting, 

20 directed according to Ihe organization's preferences and/or business preferences. 
For example, a first program could be set to record 5% of the calls performed in 
the next 24 hours while a second program could be set randomly to record 10 
interactions involving e-mail and Voice over IP (VoIP) interactions scheduled for 
a specific time interval. Following the location of the specific interactions, the 

25 interactions are made available to the reviewers through the use the evaluations 
application 182 of the application layer of Fig. 8B. The automatic scheduler 186 
will also alert the reviewers, either by e-mail. Short Message Service (SMS), 
Instant Messaging (IM), and the like, that there are interactions waiting for 
review. Using the evaluations application 182 the interaction samples are played 

30 back and reviewed and the reviewers enter the results into evaluation forms in a 
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pre-defined maimer. Subsequently, the evaluations are stored in the organization's 
database (not shown) and are available for access by the business analysis module 
190 of the business layer 188 of Fig. 8C for the purpose of collecting business 
intelligence. The module 190 could detect, for example, fiaudulent bdiavior, 
5 dealings in personal accounts, "firont run nin g", iintrutiifiil dealing with the 
organization's commodities, accepting gifts that may effect integrity and any 
unfamiliar or unusual behavioral pattem that may indicate that a conflict of 
interest influenced the judgment of the trader, and the like. 

There ^e situations where a trader fails (either vmintentionally due to 
10 excessive work load or stress or intentionally for the purposes of "front running") 
to enter the transaction details into the transaction ^stem. Such an event is 
considered illegal and financially, harmful to the organization since the situation 
of "long position" vs. "short position" (e.g. extra cash vs. shortage of cash, 
respectively) provides an distorted picture of the events. The proposed apparatus, 
15 system and method includes a cont^t analysis component that is operative m the 
•• spotting of words and/or sentences. Hie identification or detection of 
■rr-:ps such as "buy XXXXXX K$", or such as "order for me XXXX shares", 
and the lack of entering of transactional information detected by screen events 
that capture screen events on the transactional system analyzed by the system, 
20 will send suitable notification to a supervising individual, such as a Chief Trader, 
in order to provide the option of following up on the specific mteraction and in 
order to assure that the transaction is altered properly into the transaction syston. 
Optionally, a threshold value can be set during set-up to indicate the amount 
above which the alarm will be triggered or the notification will be produced in an 
25 automatic manner. After the proposed apparatus will prove to be fiiUy reliable it 
could be modified such as to provide the option of feeding directly and 
automatically the transaction system without depending on the recall capability or 
the willingness of the trader. 

The sequence of the transaction entered into the transaction system 
30 could indicate misconduct, such as "front runnmg" or "insider trading". The 
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proposed apparatus uses screen event capturing verified against the actual 
timestamp of the interaction and the captured content. For example the fact that a 
first transaction occurred before a second transaction and tiie second transaction 
was entered into the transaction system before the iSrst transaction may indicate a 
5 potential misconduct and therefore could trigger generate an alarm and/or 
generate and send a suitable notification to a supervising individual. 

The proposed apparatus, system and method provides real-time 
detection of suspicious activities, unusual customer behavior patterns, and the 
like, based on content analysis, screen events and additional data gathering 
10 mformation. For example, a customer asking for the transfer of $1000 a thousand 
times can be tagged as suspicious. Similarly, the existence of two accounts with 
different customer names and different customer addresses but with the same 
telephone number could be also considered as suspicious and tagged accordingly. 
Note should be taken that in addition to the monitoring of individual customers 
15 monitoring of institutional customers, such as mutual funds, corporate accounts, 
and the like, is provided. 

The monitoring and the reviewing of transaction-specific set Cv ; 
mteractions makes available the option of providing quality management having 
full capabilities. Quality management is constantly implemented in a various 
20 market segments, such as for example, in Call Centers where service providers 
are evaluated for their quality of service and assigned e-leaming sessions to 
promote then: skills to the benefit of the organization. The proposed apparatus, 
system and method innovatively utilize the quality management applications in 
the financial institution to promote quality of service of traders, of brokers and the 
25 like. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defined only by the 
claims, which follow. 

30 
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CLAIMS 

1. An apparatus for the synchronization, integration and association of 
financially related captured interactions, the apparatus comprising: 

an at least one first interaction capturing device for 
5 capturing an at least one interaction; 

an at least one second interaction capturing device for 
capturing an at least one second interaction; 

a synchronizing device to synchronize the at least one 
first interaction with the at least one second interaction; and 
10 an integration device to integrate the at least one first 

interaction with the at least one second interaction into an at 
least one integrated interaction. 

2. The apparatus of claim 1 further comprising an association device 
to associate the at least one integrated interaction with the at least 

1 5 one transaction* 

3. The apparatus of claim 1 further comprisi!?i . leeist one 
recording and logging device for storing the at i.r interaction, 
the at least one second interaction. 

4. The apparatus of claim 1 further comprising an at least one 
20 intCTaction retrieval device to retrieve the at least one first 

interaction, the at least one second interaction, and tiie at least one 
mtegrated interaction. 

5. The apparatus of claim 1 further comprising an. at least one 
interaction playback device to playback the at least one first 

25 interaction, the at least one second interaction and the at least one 

integrated interaction. 

6. The apparatus of claim 1 further comprising a set-up and calibration 
device to set up the parameters of the apparatus and to adaptively 
calibrate the parameters of the apparatus. 
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7. The apparatus of claim 1 further comprising a secure access device 
to provide for secure, controlled and limited access to the at least 
one first interaction, to the at least one second interaction and to the 
at least one integrated interaction. 
5 8. The apparatus of claim 7 wherein the secure access device provides 
secure time-based access. 

9. The apparatus for claim 1 further comprising a database for storing 
the at least one first int^ction. 

10. The apparatus for claim 1 further comprising a database for storing 
10 tiie at least one second interaction. 

1 l.The apparatus for claim 1 further comprising a database for storing 
the at least one first and second interaction. 

12. The apparatus for claim 1 further comprising a database for storing 
meta-data to provide the ability to retrieve the at least one integrated 

15 interaction. 

13. The apparatus of cIbt: ' rther comprising a content analysis 
module to analyze the > - Iv. of the at least one first interaction, 
the content of the at least one second interaction and the content of 
the at least one integrated interaction. 

20 14. The apparatus of claim 1 further comprises at least one knowledge 
database for storing information associated with the at least one first 
interaction or with the at least one second interaction or with the at 
least one integrated interaction. 

15, The apparatus for claim 1 further comprising a notification 
25 message-generating device for generating a notification message 

associated with the at least one first interaction or with tibe at least 
one second interaction or with the at least one integrated interaction. 

16. The apparatus for claim 1 further comprising a notification 
message-generating device for generating a notification message 

30 associated with misconduct. 
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17. The apparatus for claim 1 ftirther comprising a notification 
message-generating device for generating a notification message 
.associated with dispute resolution. 

18. The apparatus for claim 1 further comprising a notification 
5 message-generating device for generating a notification message 

associated with a request for playback. 

19. The apparatus of claim 1 fiirther comprising a business analysis 
device to collect and analyze business performance and 
information. 

10 20. The apparatus of claim 1 fiirther comprising a monitoring device 
to monitor the content of the at least one first interaction or the at 
least one second interaction. 
21. The apparatus of claim 1 fiirther comprising an evaluations device 
to evaluate the performance or level of service of a representative. 
15 22. The apparatus of claim 1 fiirther comprising a surveillance device 
>v.t and monitor a behavior pattern of a representa^^ 

apparatus of claim 22 wherein the surveillance device 
provides notification regarding the detection of or changes in 
behavior pattems. 

20 24. The apparatus of claim 1 wherein the at least one first mteraction 
and at least one second interaction are associated with at least one 
transaction. 

25. The apparatus of claim 7 wherein the secure access to the at least 
one first interaction, the at least one second interaction and the at 

25 least one integrated interaction is based on password-protected 

permission granting process. 

26. The apparatus of claim 25 wherein the secure access is a password 
based access. 

27. The apparatus of claim 25 wherein the secure access is a password 
30 and time based access. 
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28. The apparatus of claim 1 wherein the at least one first interaction, 
the at least one second interaction and the at least one integrated 
interaction represent an at least one interaction between at least two 
parties. 

5 29. The apparatus of claim 1 wherein the monitoring device, the 
surveillance device, the content analysis device provide information 
in real or offline. 

30. The apparatus of claim 1 further comprising an automatic scheduler 
device for the automatic selection of interactions to be monitored, 

10 analyzed and evaluated. 

31. The apparatus of claim 1 wherein the at least one first interaction or 
the at least one second interaction is an audio or video interaction. 

32. The apparatus of claim 1 wherein the at least one first interaction or 
the at least one second interaction is a data exchange or data capture 

15 interaction. 

33. A method for the synchronization, integration and association of 
financially related captured mteractions, the method comprising the 
steps of: 

20 capturing an at least one first interaction at a first interaction 

capturing device; 

capturing an at least one second interaction at a second interaction 
capturing device; 

synchronizing the at least one first interaction with the at least one 
25 second interaction; and 

integrating the at least one first interaction and the at least one 
second interaction into an at least one integrated interaction. 

34. The method of claim 33 fiirther comprising the step of associating 
the at least one integrated interaction with an at least one 

30 transaction. 
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35. The method of claim 33 further comprising the step of recording 
and logging the at least one first interaction and the at least one 
second interaction on an at least one recording and logging device. 

36. The method of claim 33 further comprising the step of retrieving 
5 the at least one first interaction, the at least one second interaction. 

37. The method of claim 36 further comprising the step of generating 
the at least one integrated interaction firom the retrieved at least one 
and second interaction. 

38. The method of claim 33 further comprising the step of playing back 
10 the at least one first interaction, the at least one second interaction, 

or the at least one integrated hiteraction. 

39. The method of claim 33 further coinprises the step of setting up the 
operating parameters of the method and adaptively calibrating the 
operative parameters. 

15 40. The metiiod of claim 33 further comprising the step of analyzing 
the contents of the at least one first interaction, the 'ihc at 

least one second interaction and the content v. Vi^it one 

integrated interaction. 

41. The method of claim 33 further comprising the step of generating a 
20 notification message concerning misconduct in association with the 

at least one first interaction, with the at least one second interaction, 
or the at least one integrated interaction. 

42. The method of claim 33 further comprising the step of collecting 
additional information. 

25 43. The method of claim 33 further comprising the step of analyzing 
the collected additional information. 
44. The method of claim 40 wherein the step of analyzing the at least 
one transaction and the associated at least one integrated interaction 
thereby analyzing tiie at least one interaction associated with a 

30 dispute. 



44 



wo 2004/090770 ^^^1^ |^^CT/IL2003/000300 

45. The method of claim 40 wherein the step of analyzing provides for 
* regulatory compliance. 

46. The method of claim 40 wherein the step of analyzing provides for 
quality management. 

5 47. The method of claim 33 further comprising the step of monitoring 
in real-time the content of the at least one first interaction, the at 
least one second interaction, and the at least one integrated 
interaction. 

48. The method of claim 45 wherein the step of monitoring provides 
10 for the detection of malpractice. 

49. The method of claim 33 further comprising the step of 
automatically scheduling and automatic selection of the at least one 
or two or integrated interaction to be monitored, analyzed and 
evaluated. 

15 50. The method of claim 33 wherein the synchronizing of the at least 
one first interaction with ^- \ 'rst one second interaction is a 
time-based synchronizing .ii or is a sequential-based 

synchronizing process. 

51. The method of claim 33 wherein the at least one first interaction 
20 and the at least one second interaction are identified by an 

interaction identifier. 

52. The method of claim 51 wherein the int^ction identifier is a 
unique pomt-in-time based identifier. 

53. The method of claim 51 wherein the at least one interaction 
25 identifier is a unique identifier. 

54. The method of claim 33 wherem the at least one first interaction and 
the at . least one second interaction constitute a time-based sequence 
of events. 
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